System and method for data assisted chroma-keying

ABSTRACT

The invention illustrates a system and method of displaying a base image and an overlay image comprising: capturing a base image of a real event; receiving an instrumentation data based on the real event; identifying a visual segment within the base image based on the instrumentation data; and rendering an overlay image within the visual segment.

CROSS REFERENCE TO RELATED APPLICATIONS

The present patent application is a Continuation of application Ser. No.10/215,520, filed Aug. 8, 2002 now U.S. Pat. No. 7,091,989, entitled “ASystem and Method for Data Assisted Chroma-Keying”, which claims benefitof U.S. provisional application entitled “Method and Apparatus for MixedReality Broadcast” filed on Aug. 10, 2001, with Ser. No. 60/311,477.

FIELD OF THE INVENTION

The invention relates generally to audio/visual content and moreparticularly to an apparatus and method for improved chroma-keying usinginstrumentation data.

BACKGROUND OF THE INVENTION

Typical television sport event coverage includes many video camerascovering different parts of the event. Some televised football gameshave as many as 20 video cameras covering the football field and arecapable of providing a viewpoint from many different directions.

In many televised events, it is desirable to include overlays inportions of the televised broadcast. These overlays may includegraphical or captured images such as scoreboards, games statistics,advertisements, logos, and play-by-play graphics. To produce atelevision program of a live event with overlays such as a footballgame, a large amount of manual input is typically required to create atelevision program displaying scenes of the football game in conjunctionwith the overlays placed in an appropriate position. For the overlays tobe effective, they should not block an important portion of the liveprogramming. However, positioning the overlays is typically a manuallyintensive operation.

SUMMARY OF THE INVENTION

The invention illustrates a system and method of displaying a base imageand an overlay image comprising: capturing a base image of a real event;receiving an instrumentation data based on the real event; identifying avisual segment within the base image based on the instrumentation data;and rendering an overlay image within the visual segment.

Other aspects and advantages of the invention will become apparent fromthe following detailed description, taken in conjunction with theaccompanying drawings, illustrated by way of example of the principlesof the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates one embodiment of a system overview according to theinvention.

FIG. 2 illustrates one embodiment of a system overview according to theinvention.

FIG. 3 illustrates an exemplary block diagram of the chroma-keyingsystem according to the invention.

FIG. 4 illustrates an exemplary process flow diagram according to theinvention.

FIG. 5 illustrates an exemplary process flow diagram according to theinvention.

FIGS. 6-7 show an exemplary screen shot illustrating one embodimentaccording to the invention.

DETAILED DESCRIPTION

Specific reference is made in detail to the embodiments of theinvention, examples of which are illustrated in the accompanyingdrawings. While the invention is described in conjunction with theembodiments, it will be understood that the embodiments are not intendedto limit the scope of the invention. The various embodiments areintended to illustrate the invention in different applications. Further,specific details are set forth in the embodiments for exemplary purposesand are not intended to limit the scope of the invention. In otherinstances, well-known methods, procedures, and components have not beendescribed in detail as not to unnecessarily obscure aspects of theinvention.

The invention includes a system and method for employing an improvedchroma-keying system that utilizes instrumentation data. The inventionutilizes techniques for seamlessly displaying an overlay image within abase image in response to the instrumentation data gathered by videocameras and/or sensors. For the sake of simplicity and clarity, theinvention is described with MPEG-2 being chosen as the deliverymechanism. However, any delivery mechanism suitable for use with theinvention may be utilized.

FIG. 1 illustrates a schematic diagram of one embodiment of a dataacquisition and transmission system for use with a digital televisionsystem. In this illustrated example, an event occurs at an event site110. In one embodiment, the event at the event site 110 is a televisedfootball game. However, any live event such as a sports event, aconcert, a theatrical event, and the like may be utilized.

A plurality of cameras 120 is utilized to capture visual and audiosignals of the event at the event site 110. In addition, the pluralityof cameras 120 also captures camera instrumentation data concurrentlywith the visual and audio signals. Camera instrumentation data mayinclude, for each video frame, the camera location, tilt, zoom, pan,field of view, focus setting, iris setting, and other informationrelated to the optics of each of the plurality of cameras 120.

A plurality of sensors 140 are utilized within the event site 110 tocapture performance instrumentation data. The performanceinstrumentation data describes the real event at the event site 110. Theplurality of sensors 140 may capture the performance instrumentationdata concurrently with the data camera instrumentation data captured bythe plurality of cameras 120. In this example of a televised footballgame, each football player may utilize a global positioning satelliteunit in their helmet as one of the plurality of sensors 140 to providethe performance instrumentation data in the form of the position relatedto the football player. In another embodiment, one of the plurality ofsensors 140 may include force sensor within each helmet to provide theperformance instrumentation data in the form of the force exerted on thefootball player. These specific examples of the plurality of sensors 140are shown for exemplary purposes only. Any type of sensor used tomeasure a physical aspect of the event at the event site 110 may beutilized.

An audio/visual equipment module 130 is configured to process the audiovisual signals. In one embodiment, the audio/visual equipment module 130is configured to receive the audio/visual signals from the plurality ofcameras 120.

A data acquisition module 150 is configured to process instrumentationdata. In one embodiment, the data acquisition module 150 is configuredto receive the camera instrumentation data from the plurality of cameras120 and the performance instrumentation data from the plurality ofsensors 140. Thus, the performance data collected in the dataacquisition module 150 includes both the camera instrumentation datawhich relates to particular parameters associated with the plurality ofcameras 120 while recording the event and the performanceinstrumentation data which relates to data captured by the plurality ofsensors 140 which describes aspects of the event.

The multiplex and modulate module 160 is configured to receive the audiovisual signals from the audio visual equipment module 130 and theinstrumentation data from the data acquisition module 150. In oneembodiment, the module 160 is configured to multiplex and modulate theaudio visual signals with the instrumentation data into a unified signalrelative to time. A transmitter module 170 is configured to receive theunified signal from the multiplex and modulate module 160 and totransmit this unified signal. A television 180 a shown as an exemplarydevice to receive the unified signal via the transmitter module 170.

With reference to FIG. 2, a system 200 is shown for acquiring andprocessing both audio and video signals of an event and correspondinginstrumentation data which describes physical parameters of the eventaccording to one embodiment of the invention. In one example within thecontext of auto racing, the instrumentation data may include car speed,engine performance, physical location of the car, forces applied to thecar, and the like. In other embodiments, the instrumentation data willvary with the specific application of the invention.

The instrumentation data corresponds with the audio and video signals inreal time; the instrumentation data and the audio and video signals aretemporally correlated. In one embodiment, they are temporally correlatedby the use of timestamps. In another embodiment, they may be temporallycorrelated by relative signal timing.

In one embodiment, the system 200 includes an audio/visual (NV) source210, an MPEG-2 encoder 212, a data injector 214, a real-time datastreamer 216, a carousel streamer 218, a trigger generator 220, an A/Vand data transport stream 222, a modulator 224, a transmitter 226, atuner 228, a demultiplexer 230, an MPEG-2 decoder 232, a presentationengine 234, a broadcast data handler 236, and an application module 238.Additional specific elements common in computer system such asprocessors, memory, user interfaces, system busses, storage devices, andthe like are not shown to prevent unnecessarily obscuring the aspects ofthe invention.

The components 210-238 are merely illustrated in FIG. 2 as oneembodiment of the system 200. Although the components 210-238 areillustrated in FIG. 2 as separate components of the system 200, two ormore of these components may be integrated, thus decreasing the numberof components in the system 200. Similarly, the components 210-238 mayalso be separated, thus increasing the number of components within thesystem 200. Further, the components 210-238 may be implemented in anycombination of hardware, firmware and software.

The A/V source 210 is connected to the MPEG-2 encoder 212 and providesthe MPEG-2 encoder with NV content. In one embodiment, the A/V source210 includes a video camera. However, in another embodiment, the A/Vsource 210 may also include a video cassette recorder, a digitalrecorder, or other means for providing A/V content. The MPEG-2 encoder212 receives the A/V content and encodes this content to form an encodedA/V data stream according the MPEG-2 standard which is well known in theart. In other embodiments, other A/V encoders such as MPEG-1 or MPEG-4may be utilized.

The MPEG-2 encoder 212, the real-time data streamer 216, the carouselstreamer 218 and the trigger generator 220 are connected to the datainjector 214. The real-time data streamer 216 provides the data injector214 with instrumentation data which describes and corresponds inreal-time with the A/V content from the A/V source 110. Instrumentationdata describes in real-time physical aspects or conditions thatcorrespond with the A/V content.

The carousel streamer 218 provides the data injector 214 with assets(e.g., images, audio clips, text files) related to the user interface.The trigger generator 220 provides the data injector 214 with data usedto activated predefined actions on the receiver (e.g., authoredquestions for a trivia game or poll, advertisement names for pop-up adinserts).

The data injector 214 receives incoming data from the MPEG-2 encoder212, the real-time data streamer 216, the carousel streamer 218, and thetrigger generator 220. The data injector 214 synchronizes the incomingdata such that the data from the real-time data streamer 216, carouselstreamer 218, and trigger generator 220 are timed with the correspondingencoded A/V data stream. The data injector 214 is connected to the A/Vand data transport stream 222 and feeds the synchronized data throughthe NV and data transport stream 222 to the modulator 224.

The modulator 224 receives the synchronized data. The synchronized dataincludes the encoded A/V data stream and associated instrumentation datafrom the real-time data streamer 216, carousel streamer 218, and triggergenerator 220. The modulator 224 broadcasts this synchronized datathrough the transmitter 226. The transmitter 226 may broadcast throughair, cable, phone lines, and the like.

The tuner 228 receives the synchronized data which is broadcast throughthe transmitter 226. The demultiplexer 230 is connected to the tuner 228and receives the synchronized data from the tuner 228. The demultiplexer230 separates the encoded A/V data stream from other data originallyfrom the real-time data streamer 216, carousel streamer 218, and triggergenerator 220. The MPEG-2 decoder 232 is connected to the demultiplexer230 and receives the encoded A/V data stream from the demultiplexer 230.The broadcast data handler 236 is connected to the demultiplexer. Thedata from the real-time data streamer 216, carousel streamer 218, andtrigger generator 220, is received by the broadcast data handler 236from the demultiplexer 230.

The MPEG-2 decoder processes the encoded A/V data stream and returns adecoded A/V data stream which is either identical or nearly identical tothe original A/V data stream from the A/V source 210. Similar to theMPEG-2 encoder 212, the MPEG-2 decoder 232 may be substituted with otherA/V encoders such as MPEG-1 or MPEG-4. The MPEG-2 decoder 232 isconnected with the presentation engine 234. The presentation engine 234receives the decoded A/V data stream from the MPEG-2 decoder 232.

The broadcast data handler 236 is connected to the application module138. The broadcast data handler 236 reformats the data from thetransport stream into data that the application module 238 can utilize.The data from the real-time data streamer 216, carousel streamer 218,and trigger generator 220 is received by the application module 238. Theapplication module 238 utilizes the data from the real-time datastreamer 216, carousel streamer 218, and trigger generator 220. Theapplication module 238 also interacts with the presentation engine 234.

With reference to FIG. 3, a system 300 is shown for acquiring andprocessing both audio and video signals of an event and correspondinginstrumentation data which describes physical parameters of the eventand camera-parameters according to one embodiment of the invention. Thesystem 300 includes a sensor 310, a segment identifier module 315, acamera 320, a color histogram module 325, a rendering module 330, and acompositing module 340.

The components 310-340 are merely illustrated in FIG. 3 as oneembodiment of the system 300. Although the components 310-340 areillustrated in FIG. 3 as separate components of the system 300, two ormore of these components may be integrated, thus decreasing the numberof components in the system 300. Similarly, the components 310-340 mayalso be separated, thus increasing the number of components within thesystem 300. Further, the components 310-340 may be implemented in anycombination of hardware, firmware and software.

In one embodiment, the sensor 310 and the camera 320 are configured onthe broadcast side and the rendering module 330 and the compositingmodule 340 are configured to be placed on the receiver side. However, inother embodiments, the rendering module 330 and the compositing module340 are configured to be placed on the broadcast side.

In one embodiment, the camera 320 is configured to capture both imagedata 360 and camera instrumentation data 365. The image data 360 is sentthe compositing module 340. The camera instrumentation data 365 is sentto the rendering module 330. The camera instrumentation data 365 mayinclude field-of-view data, camera position data, zoom data, and pandata of the event being captured by the camera 320. There may also bemultiple cameras within the system 300 wherein each camera is uniquelyidentified.

The sensor 310 is configured to capture performance instrumentation data370 for use by the rendering module 330. In one embodiment, an autoracing event is utilized to demonstrate various performanceinstrumentation data 370 within the system 300. In other embodiments,the system 300 may be applied to other events. For example, theperformance instrumentation data 370 may include car speed, car engineperformance parameters, forces exerted onto the car, car position, andthe like. Multiple sensors may be utilized within the system 300.

The segment identifier module 315 receives the camera instrumentationdata 365 and the performance instrumentation data 370. The segmentidentifier module 315 is configured to identify a visual segment throughthe use of camera instrumentation data and performance instrumentationdata. The visual segment refers to a part of the base image which iscaptured by the camera 320. In one embodiment, the visual segment is theportion of the base image which gets superimposed or keyed in. In otherwords, the visual segment identifies the pixels which will be affectedby the graphics insertion. Specific examples of graphics insertions areshown in the following figures.

The visual segment can be identified by coordinates in the base imagewhich can be obtained from the camera instrumentation data and theperformance instrumentation data. In one embodiment, a three-dimensionalmodel of the base image is utilized in conjunction with the dimensionsand coordinates of a surface used to display to correlate the camerainstrumentation data with the size and coordinates to insert thegraphic.

In another embodiment, the camera instrumentation data is utilized toidentify the corner points or contours of the area bounding the visualsegment in two dimensional space. In doing so, additional signals may beidentified within the area bounding the visual segment. In anotherembodiment, the insertion surface may be more complex than a simple twodimensional surface. In this case, a more complete three dimensionalmodel may be utilized to accurately define the corresponding visualsegment.

The camera instrumentation data allows the definition of the boundariesof the visual segment where the projections of graphics occurs. In thecase of a visual segment defined by a moving surface, such as a helmetof a football player, performance instrumentation data tracking themoving helmet and camera instrumentation data tracking the moving helmetrelative to the viewpoint perspective are utilized. In this specificexample, the visual segment moves with the corresponding footballplayer.

Further, even in a specific example of a stationary object being thevisual segment, such as the football field, the change in focus or zoomof the camera may change the coordinates of the visual segment. Thecoordinates of the visual segment may be constantly updated andrecalculated based on the instrumentation data.

The color histogram module 325 is configured to identify the range ofcolors within the visual segment. The color histogram module 325 is alsoconfigured to calculate the amount and frequency of the colors containedwithin the visual segment. In one embodiment, the color histogram module325 utilizes a specialized graph or plot to represent the number ofcolors and the amount and frequency of these colors within the visualsegment.

In one embodiment, the color histogram module 325 also identifies a newkey color value 375 and transmits this value 375 to the segmentidentifier module 315. In one embodiment, the new key color value 375 ischosen based on the nearest peak or nearest summit on the graphrepresenting the various colors within the visual segment. In anotherembodiment, the new key color value 375 is related to color distributionwhere parameters such as sensitivity determine how closely the colorsmust match the key color to be considered part of the color key.

In another embodiment, various alternate systems besides the colorhistogram module 325 may be utilized to identify the key color value.For example, a variety of statistical algorithms and system may beutilized to identify the key color value.

In one embodiment, the rendering module 330 receives the instrumentationdata, the key color value, and the visual segment information. In oneembodiment, the rendering module 330 generates an overlay image based onthe visual segment information. In another embodiment, the renderingmodule 330 generates an overlay image based on the key color value. Inyet another embodiment, the rendering module 330 generates an overlayimage based on the instrumentation data.

In one embodiment, the rendering module 330 is configured to generate anoverlay image that incorporates the insertion of graphics to bedisplayed in conjunction with the base image. The overlay image isrendered by the rendering module 330 in response to the visual segment,the key color value, and/or the instrumentation data.

The compositing module 340 receives the overlay image from the renderingmodule 330 and the image data 360 from the camera 320. In oneembodiment, the compositing module 340 integrates the image data 360within the overlay image. In other words, the compositing module 340blends the image data 360 within the overlay image to create a single,combined overlay image wherein the combined overlay image includes theoverlay image from the rendering module 330 combined with the image data360 which depicts a real event captured by the camera 320.

For the sake of clarity, the embodiment shown in the system 300 isillustrated utilizing the overlay image created by the rendering module330 and image data representing a single base image captured by thecamera 320. In another embodiment, multiple overlay images and imagedata representing multiple base images may be utilized to create astream of images representing a video stream. Further, this stream ofimages both overlay and base may be combined by the compositing module340.

A televised football game has been utilized as an example within variousembodiments of the invention. However, any type of live event issuitable as application for use with the invention. In a televisedfootball game, the static portions of the world model include thefootball field and surrounding stadium. The dynamic objects include thefootball players and the ball. If the instrumentation data includestracking the position of the football players, then the football playerpositions may be tracked using a technique such as inverse kinematics inone embodiment. If the instrumentation data includes tracking theparticular motions of the football players, then the football playermotions may be tracked using a technique such as joint position and/ororientation in one embodiment.

The flow diagrams as depicted in FIGS. 4 and 5 are merely one embodimentof the invention. In this embodiment, the flow diagrams illustrate theuse of the instrumentation data within the system 300 (FIG. 3).

The blocks within the flow diagram may be performed in a differentsequence without departing from the spirit of the invention. Further,blocks may be deleted, added or combined without departing from thespirit of the invention.

In Block 400, a real event is captured by a camera and/or sensor. Aseries of base images are captured by the camera and a correspondingseries of instrumentation data are captured by the camera and/or sensor.

In Block 410, the chroma keying parameters are selected by a user.Various chroma keying parameters may be selected such as a key colorvalue, spill removal, softness, garbage mate, key shrink, glossing,shadows, flare suppression, and the like.

In Block 420, the key color value is selected. In one embodiment, theselection of the color key value may be manually performed by the user.In another embodiment, the selection of the color key value may beautomated. The color key value may be defined over a finite range ofcolor values.

For example, a weatherman on television typically appears in front of amonochromatic background. During a televised production, a backgroundscene is overlayed onto the monochromatic background while still showingthe weatherman in front of the background scene if the key color valueis properly matched and has an appropriate range relative to themonochromatic background. The background scene is often a video streamshowing various map regions. However, if the key color value has a rangethat is too broad, portions of the weatherman are erroneously coveredwith the background scene. Additionally, if the key color value has arange that is too narrow, there will be holes in the background scenewhich will display the monochromatic background. Further, if the keycolor value is centered poorly, then both of these effects will occur.

In Block 430, the visual segment is identified according to the segmentidentifier module 315 (FIG. 3). Instrumentation data 480 and the keycolor value are received. The visual segment refers to the portion ofthe base image that receives a superimposed overlay. The key color valueand the instrumentation data 480 aid in identifying and tracking thevisual segment. The visual segment may be a dynamic target.

In Block 440, a color histogram is calculated for the visual segmentaccording to the color histogram module 325 (FIG. 2).

In Block 450, a new key color value is calculated from the colorhistogram. The new key color value updates the key color value accordingto the visual segment. The visual segment may be dynamic. For example,as the visual segment changes in color due to movement, shadows, varyingenvironment, and the like. The new key color value is transmitted to theBlock 430 thereby updating the key color value from the Block 420.

In Block 460, the rendering module 330 generates an overlay image.Multiple overlay images represents a video data stream. The overlayimage is configured to overlay the visual segment within the base image.In one embodiment, the overlay image may include a graphics image, acaptured image, and the like. The overlay image may take the form of anadvertisement, a scoreboard, a logo, and the like.

In Block 470, the compositing module 340 (FIG. 3) integrates and blendsthe overlay scene data stream and the base video stream in response tothe rendering process shown for exemplary purposes within the renderingmodule 330. The overlay image and the base image 485 are received.

The Blocks 400-470 are performed within the context of the segmentidentifier module 315, the color histogram module 325, the renderingmodule 330 and the compositing module 340 for exemplary purposes only.In other embodiments, the Blocks 400-470 may be performed in anygeneralized processor or any graphics specific processor.

FIG. 5 illustrates a depth layer technique for use within the system300. In Block 500, the instrumentation data is received by the segmentidentifier module 315 in one embodiment. The instrumentation data mayinclude both camera instrumentation data and performance instrumentationdata. The instrumentation data is utilized to generate a depth map. Thedepth map illustrates a layer representation of the objects within thebase image.

In Block 510, the depth map information may be utilized to pre-segment aportion of the visual segment. For example, the depth map information isutilized to identify an object that is located in front of an area inthe visual segment where the overlay image is inserted. The portion ofthe visual segment that is blocked by the object may be excluded fromthe visual segment and from the color histogram calculations.

In Block 520, the pre-segment information, the instrumentation data, andthe key color value are utilized to identify boundaries of the visualsegment.

For the sake of clarity, FIGS. 6 and 7 illustrate a particular screenshot for demonstrating one embodiment for the invention. Otherembodiments may contain variations of the particular screen shots shownin FIGS. 6 and 7 without departing from the spirit of the invention. Inthe screen shots, a televised football application is utilized. However,any live event may be utilized in other embodiments.

FIG. 6 illustrates screen shots 600 and 650. The screen shot 600includes a real scene image that is captured by a camera. The screenshot 600 includes, in part, a football field 610 and a plurality offootball players 620.

The screen shot 650 incorporates the base scene of the screen shot 600and an overlay image. The screen shot 650 includes a football field 660,a plurality of football players 670, and a plurality of graphics 680.The plurality of graphics 680 represent a team logo and are part of theoverlay image which is integrated with the base scene image.

In one embodiment, the system 300 (FIG. 3) integrates the overlay imagesuch as the plurality of graphics 680 and the base scene image such asthe football field 660 and the plurality of football players 670 via theinstrumentation data.

FIG. 7 illustrates screen shots 700 and 750. The screen shot 700includes a base scene image that is captured by a camera. The screenshot 700 includes, in part, a football field 710 and a plurality offootball players 720.

The screen shot 750 incorporates the base scene of the screen shot 700and an overlay image. The screen shot 750 includes a football field 760,a plurality of football players 770, and a plurality of graphics 780.The plurality of graphics 780 represent a team logo and are part of theoverlay image which is integrated with the base scene image. A group offootball players 775 are positioned in front of the plurality ofgraphics 780. Because of the position of the group of football players775 relative to the plurality of graphics 780, the visual segmentexcludes the area occupied by the group of football players 775.Accordingly, the plurality of graphics 780 are rendered behind the groupof football players 775 to prevent obstructing their view by a user.

In one embodiment, the system 300 (FIG. 3) integrates the overlay imagesuch as the plurality of graphics 780 and the base scene image such asthe football field 760, the plurality of football players 770, and thegroup of football players 775 via the instrumentation data. In anotherembodiment, the depth layer system as illustrated in FIG. 5 is utilizedfor ordering objects and excluding the group of football players 775from the visual segment via the instrumentation data.

The foregoing descriptions of specific embodiments of the invention havebeen presented for purposes of illustration and description. Forexample, the invention is described within the context of auto racingand football as merely embodiments of the invention. The invention maybe applied to a variety of other theatrical, musical, game show, realityshow, and sports productions.

They are not intended to be exhaustive or to limit the invention to theprecise embodiments disclosed, and naturally many modifications andvariations are possible in light of the above teaching. The embodimentswere chosen and described in order to explain the principles of theinvention and its practical application, to thereby enable othersskilled in the art to best utilize the invention and various embodimentswith various modifications as are suited to the particular usecontemplated. It is intended that the scope of the invention be definedby the Claims appended hereto and their equivalents.

1. A computerized method: capturing, by a data acquisition system, abase image of a real event and instrumentation data associated with thereal event, wherein the base image comprises a representation of adynamic object that is moving relative to a multi-chromatic backgroundin the real event, and the instrumentation data comprises globalpositioning data characterizing the dynamic object location in the realevent; transmitting the captured base image and the instrumentationdata; receiving the transmitted base image and the instrumentation data;using the received global positioning data to identify a visual segmentwithin the received base image, wherein the visual segment is part ofthe base image, excludes an entire area of the base image occupied bythe dynamic object representation, is smaller than a size of the baseimage, and includes a part of the multi-chromatic background; renderingan overlay image within the visual segment, wherein the overlay image isbehind the dynamic object representation; and simultaneously outputtingthe base image and the overlay image within the visual segment.
 2. Themethod according to claim 1, wherein the instrumentation data is furtherassociated with a camera device and includes information associated withat least one of a camera position, a camera zoom, a camera pan, a cameratilt, and a camera field-of-view.
 3. The method according to claim 1,wherein the instrumentation data further comprises performanceinstrumentation data associated with one or more physical parameters ofthe real event.
 4. The method according to claim 3, wherein theperformance instrumentation data is associated with a participant in thereal event.
 5. The method according to claim 1 further comprisingtracking a location of the visual segment within the base image usingthe instrumentation data.
 6. The method according to claim 1, whereinthe visual segment is defined by a moving surface.
 7. The methodaccording to claim 6 further comprising updating the visual segment byutilizing the updated key color value.
 8. The method according to claim1, further comprising identifying an updated key color value for a keycolor value from a color distribution of the visual segment.
 9. Themethod of claim 1, wherein the overlay image is one of an advertisement,a scoreboard, and a logo.
 10. The method of claim 1, wherein thecharacterizing data further includes data indicating a force on theobject.
 11. The computerized method of claim 1, further comprising:generating a depth map for the base image, the depth map illustrating alayer representation of the dynamic object in the base image; andpre-segmenting the visual segment to identify the dynamic object in thereceived base image using depth map.
 12. An apparatus comprising: meansfor capturing a base image of a real event, the base image comprises arepresentation of a dynamic object that is moving relative to amulti-chromatic background in the real event; means for capturing andbroadcasting instrumentation data associated with the real event by thedata acquisition system, the instrumentation data comprises globalpositioning data characterizing the dynamic object location in the realevent, wherein means for capturing and broadcasting the instrumentationdata includes a sensor; means for identifying a visual segment withinthe base image by utilizing received global positioning data, whereinthe visual segment is part of the base image, excludes an entire area ofthe base image occupied by the dynamic object representation, is smallerthan a size of the base image, and includes a part of themulti-chromatic background; means for rendering an overlay image withinthe visual segment, wherein the overlay image is behind the dynamicobject representation; and means for simultaneously outputting the baseimage and the overlay image within the visual segment.
 13. The systemaccording to claim 12 wherein the instrumentation data is furtherassociated with a camera device and includes one of a camera position, acamera zoom, a camera pan, a camera tilt, and a camera field-of-view.14. The system according to claim 12 wherein the instrumentation data isassociated with a participant in the real event.
 15. The system of claim12, wherein the overlay image is one of an advertisement, a scoreboard,and a logo.
 16. The system of claim 12, wherein the characterizing datafurther includes data indicating a force on the object.
 17. Acomputerized method comprising: receiving, by a data acquisition system,a base image of a real event and instrumentation data associated withthe real event, wherein the base image comprises a representation of adynamic object that is moving relative to a multi-chromatic backgroundin the real event, the instrumentation data comprises global positioningdata characterizing the dynamic object location in the real event, andthe base image and the instrumentation data are concurrently captured;identifying a visual segment within the received base image using thereceived global positioning data, wherein the visual segment is part ofthe base image, excludes an entire area of the base image occupied bythe dynamic object representation, is smaller than a size of the baseimage, and includes a part of the multi-chromatic background; renderingan overlay image within the visual segment, wherein the overlay image isbehind the dynamic object representation; and simultaneously outputtingthe base image and the overlay image within the visual segment.